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Abstract 

Intransitivity is supposed to be a main reason for deficits in coevolutionary progress and inheritable su¬ 
periority. Besides, coevolutionary dynamics is characterized by interactions yielding subjective fitness, but 
aiming at solutions that are superior with respect to an objective measurement. Such an approximation 
of objective fitness may be, for instance, generalization performance. In the paper a link between rating- 
and ranking based measures of intransitivity and fitness landscapes that can address the dichotomy be¬ 
tween subjective and objective fitness is explored. The approach is illustrated by numerical experiments 
involving a simple random game with continuously tunable degree of randomness. 


1 Introduction 

Despite earlier promises and optimism, using coevolutionary algorithms (CEAs) for evolving candidate 
solutions towards an optimum remains a complicated and almost arcane matter with generally unclear 
prospects of success mm 120] , This is prominently caused by a defining feature of coevolution. CEAs 
are driven by fitness that originates from interaction of candidate solutions with other candidate solutions. 
In other words, the fitness obtained from these interactions is subjective as it depends on which candidate 
solutions are actually interacting and when in coevolutionary run-time the interactions take place. The 
interactions can be understood as to constitute tests, which leads to labeling such kinds of coevolutionary 
problems as test-based problems mm- Test-based problems particularly occur in game playing or 
game-like contexts, for instance in situations where the players’ strategies are subject to (competitive) 
coevolutionary optimum finding nans]. It has been argued that in games with players and strategies the 
player space can be understood a phenotypic according to the framework of fitness landscapes pi), HJ, while 
the strategy space is genotypic mm- This view is adopted in the following discussion. 

Notwithstanding that coevolutionary dynamics is induced by subjective fitness, the aim of using a CEA 
is identifying candidate solutions that are superior in a more general sense. Hence, in coevolution next to 
the fitness resulting from (a limited number of) tests, a second notion of fitness is helpful. Such a fitness 
generalizing subjective fitness occurs in test-based problems in different forms. For games with players and 
strategies, there is usually no absolute quality measurement, or an absolute quality measurement would 
require to evaluate all possible test cases, which is computationally infeasible H 0 dSj. To circumvent 
this problem and enable experimental studies of the relationships between subjective fitness and absolute 
quality measurements, number games have been proposed, for instance minimal substrates PSl El]. These 
artificial problem settings postulate an absolute quality, called objective fitness [7]. In this line of reasoning, 
coevolutionary dynamics can be understood as aiming at progress in objective fitness by proxy of subjective 
fitness. Consequently, a main difficulty in designing CEAs stems from the question of how well subjective 
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fitness represents objective fitness. In analogy to the (postulated) objective fitness of number games, for 
games with players and strategies all general quality measurements of subjective fitness are interpretable as 
objective fitness. This implies that for game playing there may only be an approximation of objective fitness 
and different approximations are possible, for instance different instances of generalization performance 0121- 
Put another way, this interpretation suggests that in game playing there are different degrees of objective 
fitness. 

Application examples of CEAs have frequently reported experiments showing mediocre performance, 
mostly attributed to coevolutionary intransitivity [31131 EH]. Generally speaking, intransitivity occurs when 
superiority relations are cyclic. Such cyclic superiority relations have consequences for coevolutionary dy¬ 
namics as intransitivity may occurs across subsequent generations. In such a case, it may be that all solutions 
at generation k + 1 are better than at k, and that the same applies for k + 2 with respect to k + 1. This, 
however, does not imply that the solutions at A;+ 2 are strictly better than at k. Cyclic superiority relations 
that occur across generations are connoted as coevolutionary (dynamic) intransitivity. In this paper, the 
problem of coevolutionary intransitivity is linked to the dichotomy between subjective and objective fitness. 
This is done by combining a rating- and ranking-based measuring approach of intransitivity proposed by 
Samothrakis et al. [ 191 with a framework of codynamic fitness landscapes recently suggested [18] . Codynamic 
fitness landscapes enable to analyse the relationship between objective and subjective fitness for all possible 
solutions of the coevolutionary search process. The landscape approach proposed particularly explores how 
coevolutionary intransitivity is related to the objective-versus-subjective-fitness issue. The remainder of 
the paper is structured as follows. In the next section, the concept of codynamic landscapes composed of 
objective and subjective fitness is briefly recalled. In Sec. 3, intransitivity is discussed. For the discussion, a 
simple random game is introduced, where the degree of randomness can be continuously tuned. It is shown 
that intransitivity can be characterized by different types of intransitivity measures. Numerical experiments 
with the simple random game are presented in Sec. 4, and Sec. 5 concludes the paper with a summary. 

2 Coevolution, codynamic landscapes and number games 

This section focuses attention on an approach recently suggested [18) that is useful for understanding coevo¬ 
lutionary dynamics through codynamic fitness landscapes. Such landscapes allow studying the relationship 
between objective and subjective fitness which, in turn, mainly determines coevolutionary dynamics. 

We define objective fitness as the triple of search space S with search space points s G S, neighborhood 
structure n(s) and fitness function f 0 bj(s)- The objective fitness landscape can be considered as to describe 
the optimization problem to be solved by the CEA. This problem solving is based on coevolutionary interac¬ 
tions between potential solutions which yields subjective fitness. Hence, subjective fitness can be viewed as 
the way the CEA perceives the problem posed by the objective fitness. From this, it appears to be sensible 
to assume that the subjective landscape possesses the same search space and neighborhood structure, but 
has a fitness function f su b that more or less strongly deviates from the objective fitness f a bj. This can be 
seen as the subjective fitness usually overestimating or underestimating objective fitness. Moreover, for a 
coevolutionary run, the deviation between objective and subjective fitness is dynamic. In other word, the 
coevolutionary dynamics dynamically deforms the subjective fitness landscape. In the following, the link be¬ 
tween subjective and objective fitness is exemplified for a number game, which is also called a coevolutionary 
minimal substrate mm- 

In this number game a population P of players is considered that inhabits the search spaces S. The 
search space is one-dimensional and real-valued. At each instance of the game k = 0,1, 2,..., the players 
of P(k) may have possible values s € S. An objective fitness function is defined over the search space, that 
is fobj(s), which consequently casts an objective fitness landscape. The subjective fitness of P(k) is the 
result of an interactive number game. Therefore, for each calculation of the subjective fitness f su b ( s ) for a 
player from P, a sample cr(E) of evaluators from E C P is randomly selected. This sample is statistically 
independent from the sample for the next calculation. Denote /i the size of the sample a (E) out of A 
evaluators, with n < A. The number game further defines that the fitness f su b(s) with respect to the sample 
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(j{E) is calculated by counting the (averaged) number of members in &(E) that have a smaller objective 
fitness f 0 bj(cri(E)), i = 1 , 2 ,..., fi, than the objective fitness fobj(s) |18j: 

fsub(s) = i^evaI(s,<Ti(£)) with eval(s, = j J othe^wiie^ > ^ 

Note that the number game considered postulates objective fitness and defines by Eq. (jTJ) how subjective 
fitness is obtained by a coevolutionary interaction. In the next section, this perspective of subjective and 
objective fitness is applied to games with players and strategies. A population of players is engaged to 
evaluate their subjective fitness by interaction with other players. 


3 Static and coevolutionary intransitivity in games 


A relation R is called intransitive over a set S if for three elements {si,S2,S3} E S the relation (s\Rs 2 ) A 
(S 2 -RS 3 ) does not always imply siRs^. An instance of intransitivity is superiority relations that are cyclic, 
which in its most obvious and purest form appears in game playing. Cyclic superiority relations here mean 
that for three players and three strategies si, S 2 and S 3 , the player using si wins against s 2 , and s 2 wins 
against S3, but si loses against S3. A simple example is a rock-paper-scissor game, where “paper” wins over 
“rock”, “scissor” wins over “paper”, but “scissor” loses against “rock”. Thus, “paper”, “rock” and “scissor” 
are possible strategies a player can adopt in this game. Note that this kind of intransitivity is a feature of 
the preference in a single round of the game. Hence, such an intransitivity is static (and actually game- 
induced) and has no immediate link to (co-)evolutionary dynamics. Consequently, the next question is how 
these superiority relations resemble situations with coevolutionary intransitivity and can be understood by 
the dichotomy between subjective and objective fitness. To obtain (co-)evolutionary dynamics, the players 
need to adjust their strategies, and the game needs to be played for more than one round. In other words, 
studying iterated games is also interesting as it serves to juxtapose static (game-induced) intransitivity with 
coevolutionary (dynamic and search-induced) intransitivity. 

One way to build a relationship between game results and fitness is to apply a rating system. Examples 
of rating systems are the Elo system to evaluate chess players [sun], or the Bradley-Terry-Luce model of 
paired comparison mm- Recently, it has been shown that this methodology is also useful for analyzing 
coevolutionary intransitivity m- A rating system creates a probabilistic model based on past game results 
that can be seen as a predictor of future results. Most significantly, the rating system also imposes a 
(temporal) ranking of the players. In the following, these ideas are applied to a simple random game where 
the degree of randomness can be tuned. The game consists of players using a strategy to perform against all 
other players once, called a round robin tournament. The game outcome, which can be interpreted as payoff, 
is subject to the players’ ratings and random. Given that the game has N players, there are 1 ' > games in 
a single round robin. Define a percentage p ran d of games that have a random result to obtain p ran d ■ ^ 

games whose outcome is chance with a predefined distribution. The remaining games end deterministically 
according to the rating difference between the players. Thus, such a game falls into the category of perfect 
and incomplete information. Viewed over a series of round robin tournaments, this interaction between 
rating-based determinism and random chance creates temporal “rating triangles”, where a player scores 
high results (and has a high rating) over a certain time, but may also lose against a nominal weaker (low- 
rating) player, which over time may or may not show the same characteristics towards a third player. Such 
a behavior complies with coevolutionary (dynamic and actually search-induced) intransitivity. In addition, 
the game can also reproduce rock-paper-scissor-like intransitivities. For N players the maximal number of 
static intransitivities is 


Trintra i 


JV \ _ (N-2)(N- 1 )N 
3 ) ~ 6 


( 2 ) 


see m m- Any three players form a triangle of cyclic superiority relations if they each win one game 
against the other two. Thus, for N not very large, the (average) number of actual static intransitivities 
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Table 1: Results of the simple random game for three instance of the game 


# 

rt(0) 

#1 

#2 

#3 

#4 

#5 

sc(0) 

rt(l) 

#1 

#2 

#3 

#4 

#5 

sc(l) 

rt(2) 

gP 

rank 

sc 

1 

1600 

X 

1 

1 

1 

1 

4 

1630 

X 

0 

1 

1 

1 

3 

1642 

3.5 

1 


2 

1600 

0 

X 

1 

0 

1 

2 

1600 

1 

X 

1 

1 

0 

3 

1615 

2.5 

2 


3 

1600 

0 

0 

X 

1 

1 

2 

1600 

0 

0 

X 

0 

0 

0 

1570 

1 

4.5 


4 

1600 

0 

l 

0 

X 

1 

2 

1600 

0 

0 

l 

X 

1 

2 

1600 

2 

3 


5 

1600 

0 

0 

0 

0 

X 

0 

1570 

0 

1 

l 

0 

X 

2 

1573 

1 

4.5 



can be determined by enumeration and gives a static intransitivity measure called the intransitivity index 
(itx) [6] [T9] , As an alternative, Samothrakis et al. [19] suggested to use a difference measure based on 
Kullback-Leibler divergence (kid) between the prediction made by the rating system and the actual outcome 
to measure static intransitivity. Both quantities itx and kid are subjects of the numerical experiments 
reported in the next section. 

A game in a coevolutionary setting involves finding the strategy a player should adopt to score best 
according to a given understanding of performance. This clearly implies that the performance measurement 
should generalize a single round robin tournament. Thus, if there are several instances of round robin 
tournaments, the overall results can also be accounted for by generalization performance mm- Each instance 
of a round robin can be scaled to a generation of coevolutionary run-time. Generalization performance is 
defined as mean score of a solution in all possible test cases. Because considering all possible test cases may 
be computationally infeasible, Chong et al. mm used a statistical approach involving confidence bounds to 
estimate the amount of needed test cases for a given error margin. Given this understanding, and assuming 
that all strategies are equally likely to be selected as test strategies, the generalization performance of 
strategy i is: 


gPi = 


^ k=1 


(3) 


where sc i{k) is the score the i-th strategy yields in the k -th instance of a round robin tournament. The needed 
number of instances K, depends on the bounds given by Chong et al. mm ■ Generalization performance also 
builds a relationship between actual game results and fitness and therefore can be seen as an alternative to 
a rating system. 

As an example of the simple random game assume that there are five players (N = 5), denoted as #1 to 
#5, which each act upon a unknown, but internally adjusting strategy. Further assume that as an a-priori 
evaluation all players are considered equal and have the same rating, say rt(0) = 1600. These players are 
now engaged in a round robin tournament, where a win scores 1 and a loss counts 0. The results achieved 
depend on the pre-game rating and random. Assume that these results were scored, see Tab. [lj column 
3-7. While player #1 wins all games and #5 loses all games, the players #2-#4 build a rock-paper-scissor 
triangle of cyclic superiority relations. These results somehow violate the expectations established by the 
initial (pre-game) rating, which could have been met by all players winning 2 out of 4 games. Furthermore, 
the results show clearly that the game cannot be completely deterministic with respect to the a-priori 
evaluation. In other words, a rating and ranking approach subsumes the game history and can only be 
a predictor of future game results. The quality of prediction depends on the percentage of random game 
results. Also, from the results of the round robin, it is not evident whether player #1 was as successful 
as it was because its strategy was (objectively) good, or because the strategies of the other players were 
(objectively) poor. All, the round robin gives is a comparison. 

The a-priori rating were indicating that all players have the same rank (and hence there is no ranking 
difference between them), but the results were showing otherwise. Hence, the round robin tournament 
updates the rating, producing an after-game rating. According to the Elo system mm, which is adopted 
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here, this is done via first calculating the expected outcome 


ex »(^) 5-/ i _|_ ]^Q( r tj (k)—rti(k ))/400 

3 

The quantity exj(/c) summarizes winning probabilities of player $7 with respect to all other players in round 
k. For a single game, that is j = 1, the quantity ex* is the expected winning probability for the player 
winning against player #j. For the example with all players having the same rating, the expected outcome 
is also the same, namely ex* = 2. The rating of the players is updated according to the difference between 
expectation and actual score: 


( 4 ) 


rt i(k + 1) = rti(fc) + K(sci(k) - exj(fc)). (5) 

The K is called the K— factor, which tunes the sensitivity between the rating and the results of a single 
round robin tournament. Using K = 15, the new rating rt(l) gives differences between the players, see 
Tab. [lj column 9. The best player #1 is now ranked highest, the poorest player #5 is ranked lowest. 
Also note that the players #2-^4 engaged in the game-induced intransitivity triangle still share the same 
rating as before. Now assume the next round of the game. The strategies of the players have been adjusted, 
supposedly by a (competitive) coevolutionary search process. See the results in Tab. [lj column 10-14. The 
outcome generally confirms the impression of the first round with player $T still being strong. Violating 
the expectations are the good scores of the players #2 and #5, and the poor score of player #3. Note that 
in this round the players #1, #2, and #5 as well as the players #2, #4, and #5 form a rock-paper-scissor 
triangle of game-induced intransitivity. Contrary to the first round, these players now neither have the same 
score, nor the same pre-game or after-game rating. This after-game rating rt(2) is calculated according to 
Eq. ([5]) and given in Tab. [lj column 16. 

As there are now two instances of the round robin tournament, an first estimation of the generalized 
performance gp can be obtained by averaging the scores according to Eq. ([ 3 ]). The results are given in Tab. 
[lj column 17. Conforming with this account of quality evaluation, player #1 is best, and can be ranked first, 
followed by players #2 and #4. The ranking with respect to generalization performance is given in Tab. [lj 
column 18. Note that the ranking gives average ranks for tied ranks as for player #3 and #5 (gp = 1 leads 
to rank = 4.5), which preserves the sum over all ranks. Further note that this ranking is almost equal to 
the ranking according to ratings, with the exception of players #3 and #5 which have a very similar but 
not equal rating. 

Now, the simple random game is interpreted according to the landscape view of subjective and objective 
fitness. Recall that subjective fitness is associated with fitness gained by individuals through interaction 
with others. According to this view, a round robin tournament yields subjective fitness: f su b = sc(k). 
Objective fitness, in turn, generalizes subjective fitness in terms of an absolute quality measurement. Possible 
candidates are the rating f 0 bj = rt(k) or generalization performance f a bj = gp(k). Defining subjective and 
objective fitness in such a way also gives raise to reformulating coevolutionary intransitivity. Generally 
speaking, coevolutionary intransitivity involves cycling of (objective) solution quality. This cycling may be 
caused by subjective fitness not adequately representing objective fitness. Hence, subjective fitness may 
drive evolution into search space regions visited before but evaluated differently, or generally into directions 
not favorable. Hence, coevolutionary (dynamic) intransitivity can be understood as temporal mismatches 
in order between subjective and objective fitness. Consider again the example of the game whose results are 
given in Tab. 1. Suppose another instance of the round robin is played, and (omitting the specific results) 
the scores are in Tab. 1, column 19. The rating rt(k) is considered to be objective, while sc(k) is subjective. 
For player $T, the rating is rti = 1600 < 1630 < 1642, while its score is sci = 4 > 3 > 1, which is a 
temporal mismatch between objective and subjective fitness. If we were to suppose for a moment that the 
rating declared as objective fitness is indeed the quantity to achieve in coevolutionary search, and if a CEA 
were to use score for guiding this search, then player #1 would likely be misguided. On the other hand, 
for player #2, the rating rt 2 = 1600 < 1630 < 1642, and the score SC 2 = 2 < 3 < 4 show a match between 
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(d) (e) (f) 

Fig. 1. Scores, ratings, and generalization performance for two percentages of random games: (a)-(c), 
Prand — 0.01, (d)—(f), Prand — 0.75. 


subjective and objective fitness. These conditions can be reformulated employing a ranking function with 
tied ranks and gives a measure of coevolutionary intransitivity. Hence, the player-wise temporal mismatch 
(ptm) can be defined as the average number of rank mismatches: r&nk(f 0 bj(k), f 0 bj{k + 1 )),fobj{k + 2)) / 
rank (f su b(k), f su b(k + 1)), f su b{k + 2)) for each player for a given number of instances of round robins. 

An alternative measure of coevolutionary intransitivity that is related to the ptm just discussed stems 
from the fact that coevolutionary selection is based on comparing subjective fitness values. Hence, the fitness 
ranking within one instance of the game (or one generation) gives indication as to what direction is preferred. 
If difficulties in the search process are caused by how well subjective fitness represents objective fitness, 
then the difference in the ranking according to subjective fitness and the ranking according to objective 
fitness for each instance is also a suitable measure of coevolutionary intransitivity. Therefore, the quantity 
|rank (f su b(k)) — rank(/ 0 &j(fc))| over all players for each instance k is another measure of coevolutionary 
intransitivity and is called collective ranking difference (crd). The observations discussed so far suggest 
some relationships, namely that game-induced, static intransitivity has ambiguous effect on coevolutionary 
progress, and that coevolutionary, dynamic intransitivity can be expressed as ranking differences between 
objective and subjective fitness. In other words, the quantities ptm and crd may be useful as measures of 
coevolutionary intransitivity. All these relationships can be studied by numerical experiments, which are 
the topic of next section. 
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sc rt gp 


(a) (b) (c) 



(d) (e) (f) 


Fig. 2. Intransitivity measure itx versus score = sc(k), rating rt(k) and generalization performance gp(k) 
(a-c) as scatter plots; relationships between the intransitivity measure itx, kid, and ptm (d-f) 

4 Numerical experiments 

The report of experimental results starts with the time evolution of the simple random game introduced 
in the last section. Fig. 1 shows the scores, ratings and generalization performances for two different 
percentages of random results ( p ra nd = 0-01 and p ra nd = 0.75) and N = 6 players and 20 instances of 
a round robin. The experiments are initialized with ratings rt(0) being slightly spread around the value 
rt(0) = 1600. The randomly determined chance to win or to lose is evenly distributed. The figures show 
curves for a single instance of the randomly determined part of the game outcomes. Hence, the curves are 
not meant to be statistically significant, but for illustrating typical behaviour only. It can be seen that for 
a low number of random outcome (Fig. la,b,c) the scores each player achieves are mainly defined by the 
initial rating. Small differences in the initial rating are amplified and lead to well-sorted long-term rankings 
of both the rating and generalization performance. For a large number of random game results (Fig. ld,e,f) 
the scores are almost purely chance which is evenly distributed. Consistently, ratings and generalization 
performances tend to approach the expected value implied by the underlying distribution for all players alike. 
The next experiments address the relationships between static (game-induced) intransitivity expressed by 
the measure itx and quantities representing subjective as well as objective fitness. The scatter plots given in 
Fig. 2a,b,c are for IV = 16 players and five levels of prand- The experimental setup includes a repetition of 
each run for 100 times for 1000 instances of the round robin, where the first 200 instances are discarded to 
omit transients. Note that this gives a sufficient number of instance according to the bounds of generalization 
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crd(sc,gp) max sc max gp 


(a) (b) (c) 



(d) (e) (f) 


Fig. 3. Intransitivity measure crd versus time average itx, max sc, and max pg (a-c) as scatter plots; 
relationships between the intransitivity measure crd, itx, and ptm (d-f) 


performance mm- Hence, the results can be seen as statistically significant. In fact, the 99% confidence 
intervals are so small that they are not depicted in the figures. It can be seen that for each level of 
randomness in the game, we obtain a distinct level of static intransitiviy measured by itx, where rising p ra nd 
also increases itx. What is interesting is that there is almost no variation in score, rating or generalization 
performance for a given level of intransitivity itx. This indicates that static intransitivity seems to have little 
influence on neither subjective nor objective fitness. This is particularly visible for low levels of p ra nd where 
these is a clear sorting according to the range of fitness a given player achieved but this is not connected to 
differences in the itx. A next experiment explores the relations between the index based measure itx and the 
probabilistically motivated measure kid, see Fig. 2d. Here as well as in the following figure the results are 
for four different number of players N (N = 8,16, 24, 32) and five levels of random results p ra nd\ the time- 
average itx, denoted (itx), is given, the quantities are normalized according to the number of players. It can 
be seen that between both quantities there is a proportional relationship, which on the one hand generally 
confirms the result in [19], but does not show that itx is more brittle that kid. For the studied game it can be 
concluded that both quantities are interchangeable. Next, the relationship between the static intransitivity 
and the dynamic intransitivity measure player-wise temporal mismatch (ptm) is studied, Fig. 2e,f. For the 
mismatch ptm based on rating as objective fitness (Fig. 2e) there is a linear relationship, at least for small 
values of Prand: f° r the mismatch ptm based on generalization performance (Fig. 2f), no sensible conclusions 
about relations can be drawn. Finally, we focus on the rank-based dynamic intransitivity measure collective 
ranking difference (crd), see Fig. 3. The Fig. 3a,b,c again shows scatter plots, now for the crd based on 
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score as subjective fitness and generalization performance as objective fitness for IV = 16 players and five 
levels of Prand ■ It can be seen that although different p ra nd give different itx, there is no difference in the 
crd, Fig. 3a. However, there is a almost linear relation between the measure crd and max sc and max 
gp, at least for lower levels of p ra nd ■ This can be interpreted as the crd scaling with the time evolution of 
the subjective and objective fitness, but not with the time evolution of static intansitivity, compare to Fig. 
2a,b,c, which does not show such a characteristics. For the time-averages, there is a scaling for different 
number of players and different levels of randomness p ra ndi see Fig- 3d. In Fig. 3e, the relation between 
the crd based on rating as objective fitness (crd(sc,rt)) and the crd based on generalization performance as 
objective fitness (crd(sc,gp)) is shown. It can be seen that both quantities scale piece-wise linear for p ran d 
not very large, which allows to conclude that both quantities account for the same intransitivity properties. 
Finally, the relation between the crd and ptm is shown, Fig. 3f. It can be seen that the ptm scales weaker 
than the crd, particularly for a small number of players and high randomness, and it can be conjectured 
that the crd is a more meaningful coevolutionary intransitivity measure than the ptm. 

5 Conclusions 

This paper is a contribution to the ongoing discussion about the effect of intransitivities on coevolutionary 
progress. An approach was presented that allowed to link a rating- and ranking-based measuring approach 
of intransitivity m with a framework of fitness landscapes to enable analyzing the relationship between 
objective and subjective fitness. For experimentally illustrating the approach a simple random game with 
continuously tunable degree of randomness was proposed. Apart from the random, the game results depend 
on the ratings of the players, which reflect the past success of each player. Thus, the game proposed 
characterizes many real-world games as their outcome is also a function of chance as well as of predictions 
based on game history. 

For studying the effect of intransitivity, measures were explored. In extension of existing static intransi¬ 
tivity measures, dynamic measures that can account for coevolutionary intransitivity were proposed. These 
measure are based on rankings between subjective and objective fitness and it was shown that coevolution¬ 
ary intransitivity can be understood as a ranking problem, and hence be accounted for by ranking statistics. 
To enlarge the scope of the presented approach, as a next step the intransitivity measures could be studied 
for other types of game for instance social games as the iterated prisoner’s dilemma or board game such as 
Othello. 
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